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One  aspect  of  tutoring  skill  for  technical  subjects  i3  individualized,  adaptive  sequencing  of  the 
problems  given  to  students  as  learning  exercises.  A Curriculum  Information  network  (CIN)  describes 
the  relationships  between  the  problems  In  a CAI  curriculum  and  the  concepts  and  skills  that  they 
are  intended  to  teach.  It  is  a basis  for  selecting  problems  for  each  student  with  respect  to  his 
evolving  knowledge  of  those  concepts  and  skills.  This  paper  describes  the  application  of  a 
semantic  network  to  represent  the  complex  interrelaticnsliips  among  the  skills  in  a CM  for  the 
BAGIC  Instructional  Program,  a CAI  problem-solving  laboratory  for  introductory  programing  in  the 
BASIC  language.  The  semantic  network  is  used  in  drawing  complex  inferences  about  the  student's 
state  of  knowledge  and  the  problems  that  are  appropriate  to  present  to  him.  Such  inferences  enable 
more  skillful  problem  sequencing  by  the  CAI  system. 


1.  Introduction 


Since  it3  inception,  a major  goal  of  ccnputer- 
asslsted  Instruction  (CAI)  has  been  to  Individual- 
ize instruction  by  making  the  teaching  system's 
behavior  contingent  on  a st.udent'6  responses  during 
prior  interactions  with  the  system.  Some  early 
adaptive  CAI  systems  applied  features  of  mathe- 
matical learning  and  decision  theories  to  sequen- 
cing drtll-and-practice  curricula  in  elementary 
mathematics  (Suppes  & Mornings  tar,  1972)  and  in 
Initial  reading  (Green  & Atkinson,  1966).  These 
systems  generated  problems  of  different  types 
(e.g.,  addition  problems  with  or  without  "carries") 
according  to  an  estimate  of  the  student's  mastery 
of  each  type  based  on  his  prior  history  of  correct 
and  incorrect  answers.  As  a result,  more  capable 
students  could  progress  rapidly  to  harder  problem 
types  and  remediation  and  review  could  be  deter- 
mined with  respect  to  individual  difficulties 
manifested  with  specific  problem  types. 

More  recently,  features  adapted  from  Artifi- 
cial Intelligence  (AI)  systems  have  been  applied 
to  Individualize  CAI  along  other  dimensions. 
Research  on  question  answering  and  natural  language 
has  enabled  systems  such  as  SCHOLAR  (Collins, 
l/amock.  Sc  Fassafiune,  1975)  to  conduct  spontaneous 
instructional  dialogues,  giving  students  great 
freedom  of  expression  and  providing  exposure  to  a 
topic  according  to  the  student's  demonstrated 
familiarity  with  it.  Other  systems,  for  example, 
SOPHIE  (Brown  & Burton,  1975) » have  been  designed 
to  monitor  student  problem-solving  attempts  and 
react  to  errors  or  requests  for  help  with  comments 
that  reflect  the  context  In  which  the  student  has 
been  working.  Gigntficant  progress  has  teen  made 
in  AI-baced  C/I  toward  understanding  the  student's 
behavior  with  respect  to  representations  of  the 
knowledge  he  acquires  and  uses,  and  not  merely  in 
terms  of  his  performance  on  particular  questions 
end  problems. 

Our  research  over  the  past  few  years  has 
attempted  to  apply  some  AI  techniques  to 
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individualized  curriculum  sequencing  for  complex 
scientific  and  technical  subjects.  Traditionally, 
instruction  in  these  types  of  subjects  has  depended 
heavily  on  students  solving  large  numbers  of  prob- 
lems. This  method  has  proven  to  be  effective  for 
forcing  the  integration  of  concepts  and  problem- 
solving  skills  described  In  lectures  and  readings. 
Our  ccncem  has  been  to  individualize  the  sequen- 
cing of  a curriculum  of  complex  problems  in  order 
to  Improve  the  acquisition  and  integration  of  the 
underlying  concepts  and  skills  of  the  subject.  The 
result  of  our  Initial  efforts  was  the  concept  of 
the  Curriculum  Information  Network  ( CIU) , a means 
of  describing  the  relationships  among  the  problems 
in  a curriculum  in  terms  of  procedural  skills 
involved  in  solving  them,  and  for  modeling  student 
learning  with  respect  to  those  same  skills  (Barr, 
Beard,  Sc  Atkinson,  1976).  A CIN  and  a problem- 
selection  procedure  were  Implemented  In  the  BASIC 
Instructional  Program  (BIP),  a CAI  system  that 
teaches  the  BASIC  programming  language  at  an  Intro- 
ductory level.  The  present  paper  describes  our 
recent  research.  In  which  we  developed  a semantic 
network  representation  for  structural  and  peda- 
gogical knowledge  about  BASIC  and  some  Inference 
techniques  for  using  this  network  to  increase  the 
sophistication  of  CIN-based  problem  selection  in 
the  DIP  system. 

2.  O'/e rvlcw  of  the  FTP  system 
2.1  resign 

BIP  Is  designed  to  teach  elementary  program- 
ming concepts  and  skills  without  lectures  or  a 
standard  textbook.  All  of  the  resources  available 
to  students  reside  within  the  BIP  system,  illus- 
trated in  Figure  1,  except  for  a manual  written 
especially  for  the  course.  Furthermore,  after  a 
brie f interactive  introductory  lesson,  HIP  presents 
no  further  text  lessons  on  program-. tng;  Instead, 
the  student  learns  to  program  by  solving  program- 
ming problems  using  a BAGIC  interpreter  built  into 
the  instructional  system.  For  the  purposes  of  our 
research,  tie  value  of  IIP  us  a stund-ul<  r.e  CAI 


resource  1b  that  the  effects  of  problem  sequencing 
on  student  performance  are  emphasized  und  can  be 
evaluated  core  directly  than  they  could  in  a multi- 
faceted system  involving  lectures,  readings,  nr.d 
questlon-snd —answer  sessions  in  addition  to  a 
problem-solving  laboratory. 


Figure  1.  A schematic  representation 
of  the  BIP  CAI  system. 


2.2  Curriculum 

The  only  parts  of  the  HCP  system  that  ve  need 
to  consider  in  this  paper  are  those  involved  in  its 
curriculum-sequencing  process'.  The  curriculum 
consists  of  about  100  human-authored  programming 
problems,  which  we  refer  to  as  tasks.  While  BIP's 
task-selection  procedures  could  work  as  well,  or 
better,  if  tasks  were  generated  dynamically,  at 
present  there  13  no  way  to  generate  complex  tasks 
that  are  as  realistic  and  motivating  as  those 
written  by  a person.  Furthermore,  in  a real  in- 
structional situation,  it  is  the  more  usual  case 
that  a teacher  starts  with  a set  of  tasks  he  feels 
are  appropriate  and  then  considers  a task- 
sequencing strategy  rather  than  vice-versa.  The 
following  is  a typical  problem  from  the  BIP  cur- 
riculum : 

On  the  first  day  of  Christmas,  someone's 
tree  love  sent  him/her  a partridge  in  a pear 
tree  (one  gift  on  the  first  day).  On  the 
second  day,  the  true  love  sent  two  turtle  doves 
in  addition  to  another  partridge  (three  gifts 
on  the  second  day).  This  continued  through 
the  12th  day,  when  the  tree  love  sent  12 
lords,  11  ladles,  10  drummers,  . . . all  the 
way  to  yet  another  partridge. 

Write  a program  that  computes  and  prints 
the  number  of  gifts  sent  on  that  twelfth  day. 
(This  is  net  the  seme  as  the  total  number  of 
gifts  sent  throughout  all  12  days — Just  the 
number  sent  on  that  single  12th  day.) 

2.3  The  CIH:  Relating  tasks  and  nkllla 

The  primary  knowledge  base  in  EIP  is  a Cur- 
riculum Information  Network.  In  the  CIH  of  BIP- I, 
the  version  described  by  Barr,  at  al.  (1976),  each 
task  in  the  curriculum  in  described  in  terms  of  a 
set  of  procedural  skills  necessary  to  complete  it 
successfully  (l.o.,  to  write  the  P.'VllC  program 
that  is  culled  for).  About  90  skills  are  used  to 
describe  the  entile  BIP  curriculum,  with  seme  tasks 
involving  ns  few  as  one  or  two  skills,  and  others 
more  than  ton.  Each  student's  progress  three. ;h  the 
course  is  Modeled  with  respect  to  his  learning  of 
the  total  set  of  skills.  The  skills  defin'd  in 
BIP-T  in  r.o  way  constitute  a sufficient  basis  for 


representing  the  complete  process  by  which  h 6tudent 
understands  a problem,  determines  a solution  algo- 
rithm, and  implements  it  as  acceptable  BASIC  code. 
The  skills  relate  only  to  tie  ceding  process,  many 
corresponding  to  a single  line  of  code.  For 
example.  If  a problem  solution  includes  the  llr.e3 

70  INPUT  T 
80  PRIKT  R * T 


then  its  description  In  the  CIN  would  Include  the 
skills,  "assign  numeric  variable  by  -IHPUT-”  and 
"print  numeric  expression  (operation  on  variables)". 

Although  the  skills  are,  for  the  most  part, 
defined  In  terras  of  syntactic  constructs,  BIP-I 
task  selection  does  not  reflect  the  student's 
knowledge  of  syntax,  but  instead  depends  on  his 
knowledge  of  the  semantics  and  pragmatics  for  using 
the  skill  appropriately.  All  purely  syntactic 
errors  are  detected  immediately  by  the  BIP-I  inter- 
preter, which  responds  with  explicit  feedback 
describing  the  error  and  illustrating  syntactically 
correct  examples  cf  the  construct.  These  syntactic 
errors  do  not  affect  the  model  of  the  student  main- 
tained by  BIP-I.  This  model  is  affected  by  logical 
errors  which  allow  the  student's  program  to  run, 
but  not  to  produce  correct  results.  Many  of  these 
errors  can  be  associated  with  semantically  or 
pragmatically  inappropriate  use  of  the  syntactic 
constructs  described  by  BIP-I  skills. 

In  BIP-I,  the  skills  are  grouped  into  about  a 
dozen  non-overlapping  sets  called  techniques,  such 
as  simple  printing,  assignment,  nr.d  conditional 
branching.  The  techniques  themselves  are  linearly 
ordered  according  to  Judgments  about  the  relative 
complexity  of  the  skills  they  contain.  The  tech- 
nique ordering  is  used  in  BIP-I  as  a constraint  cn 
the  order  in  which  major  concepts  are  introduced 
and  ns  a scale  for  deteiT.-j.nin3  whether  problems 
available  for  remediating  particular  skills  are 
appropriate  to  the  student's  overall  progress. 

2.4  Using  a CIH  for  task  selection 

The  general  paradigm  for  applying  a CIN  to 
problem  selection  is : 

1.  From  the  model  of  the  student's  learning 
of  all  the  skills,  assemble  sets  of  skills 
that  correspond  to  relevant  pedagogical 
considerations,  such  as  "need  further 
work,"  "ready  to  be  learred,"  "already 
learned,"  etc. 

2.  Using  these  sets  and  other  historical  data 
(e.g.,  tho.-e  tasks  that  this  student  has 
already  completed),  search  the  CIN  for  the 
task  that  uses  a net  of  skills  most  con- 
gruent 'to  a sot  that  tost  satisfies 
overall  pedagogical  goals.  For  example, 
given  the  overall  goal  of  reducing  the 
total,  number  of  tasks  students  must  com- 
plete to  i’-> eh  a level  of  competence,  one 
might,  '-hc  -.e  to  search  for  a task  ulth  the 
most  ' Hirt’.ier  work"  and  "ready  to  l/r 
learn  *d"  a ills  end  Iho  IVvest  "already 
learned"  skills. 


3.  Present  tlVt  tank  to  1 1 • student  and  ana- 
lyse hi o j*’ rfuminct!  ..n  It  to  upde.lo  the 
model  oi’  his;  learning  of  the  skills. 

Three  main  theoretical  entitle:;  hear  cn  this  task- 
celeoticn  paradigm*  First,  them  la  the  represen- 
tatlen  of  knowlc  ire  to  be  learned,  which  It.  ihe 
F.kllls  In  the  CTli.  Second,  there  is  a theury  of 
learning,  which  naps  performance  on  tanka- onto 
changes  in  the  student's  knowledge  of  the  .kills. 
Finally,  there  is  a theory  of  Instruction,  which 
for  any  state  of  knowledge  determines  the  next 
task  that  i3  "best"  for  the  student  to  work  on. 

3.  Applying  a network  representation  £f  knowledge 
to  task  selection 

Our  initial  use  of  a CIN  in  BXP-I  for  selec- 
ting tanks  has  demonstrated  the  successful  applica- 
tion of  the  CIH  paradigm  (Barr,  et  al.,  1976). 
However,  the  representation  of  programming 
knowledge  and  the  model  of  student  learning  used 
In  HTP-I  are  very  rudimentary.  Most  obviously, 
BIP-I’s  grouping  of  skills  into  techniques  is  an 
oversimplification  of  the  actual  interrelations 
between  skills.  The  technique  groups  do  not  pro- 
vide a sufficient  basis  for  anticipating  a 
student's  performance  in  new  contexts  based  on  his 
performance  in  related  contexts — an  important 
aspect  of  a human  tutor's  skill  in  selecting  tasks 
for  his  student.  Likewise,  the  student  model, 
consisting  of  counters  for  each  skill,  does  not 
differentiate  various  levels  of  skill  mastery  in- 
dicated by  the  amount  of  difficulty  a student 
encounters  in  completing  tasks.  We  therefore 
undertook  to  design  a new  CIN- baaed  task-selection 
procedure  for  a EIP-II  system,  incorporating  both 
a more  detailed  representation  for  the  knowledge 
underlying  the  curriculum  and  more  complex  assump- 
tions for  modeling  student  learning.  The  remainder 
of  this  paper  will  focus  on  the  new  representation 
ve  have  developed  and  Jiow  it  is  used  in  the  BIP-II 
system. 

In  considering  alternative  representations 
for  the  knowledge  underlying  a task,  we  recognized 
that  the  most  powerful  approach  would  be  a pro- 
cedural representation  sufficient  to  synthesize 
task  solutions  (see,  for  example.  Brown,  Burton, 
Miller,  deKleer,  Purcell,  Hausmann,  & Bobrov, 

1975,  and  Carr  & Coldstein,  1977).  However,  the 
state-of-the-art  in  program  synthesis  and  analysis 
techniques  has  not  yet  advanced  to  a point  where  a 
manageable  system- could  be  implemented  for  auto- 
matically solving  programming  problems  like  those 
in  the  BIP  curriculum.  Thus,  ve  decided  to  extend 
the  original  concept  of  a set  of  skills  by  embed- 
ding the  skills  in  n network  representation 
describing  the  structural  and  pedagogtcally  sig- 
nificant relations  between  them.  The  network 
relationships  allow  Inferences  that  potentially 
add  sophistication  to  both  the  process  of  task 
selection  and  of  Interpreting  student  performance 
to  update  the  student  model.  For  example, 
unlearned  skills  that  are  deemed  to  tc  ana  gous 
to  other  skills  that  are  already  learned  can  be 
given  lower  priority  for  inclusion  In  the  next 
task  to  be  presented.  Or,  Lf  such  skills  occur 
in  a task  that  a student  quits,  then  they  can  lo 
taken  as  less  likely  sources  of  his  difficulty 
than  unlearned  skills  that  arc  analogous  to  other 


skill.;  that  an;  il  ready  known  to  ho  troubles*-**?  for 
that  student. 

3.1  The  PJSICNKT 

Rathe- r than  basing  the  network  of  knowledge 
to  to  .learned  directly  on  the  HCP  curriculum,  we 
telle  the  skill  relationships  on  n general  repre- 
sentation for  BASIC  programs.  From  a general 
analysis  of  the  BASIC  language,  guides  to  BASIC 
programming,  and  the  skills  arid  techniques  of 
LIP-I,  we  developed  a network  representation  for 
BASIC  programming  constructs  ( the  RASICIIET) , a 
simplified  portion  of  which  is  shown  in  Figure  2. 


contrgl'structure 


UNCONDITIONAL  — CONOIIICNAL 


NEXPR  REL  NEXPR 


Figure  2.  A simplified  portion  of  the  BASICNET 
describing  control  structures  of  the 
BASIC  language. 

The  node  names  are  self-explanatory;  the  links 
(relationships)  are  Kind,  Component , Hardness , and 
(mutual  functional)  Be pendency.  The  section  of 
the  BASICNET  shown  specifies  that  there  are  two 
kinds  of  control  structures,  and  expresses  a Judg- 
ment that  the  conditional  kind  is  harder  to  learn 
than  the  unconditional.  There  are  two  kinds  of 
conditional  structures,  and  FOB'IEXT  is  harder 
than  IF'THEN.  The  components  of  an  IF 'TIEN  state- 
ment are  the  words  "IF"  and  "THEN"  with  the  Boolean 
condition  and  the  line  number  in  the  appropriate 
places.  For  the  purposes  of  this  illustration, 
the  DOOLEAil  consists  of  a numeric  expression 
(NEXPH),  a relational  operator  (RF.L),  and  another 
NEXPR;  among  the  three  kinds  of  NEXPRs,  numeric 
literals  (KLIT)  are  easiest,  and  numeric  variables 
(tiVAR)  and  simple  arithmetic  expressions  (SIMARITIl) 
are  increasingly  hard. 

Note  that  the  downward  links  in  the  figure 
provide  information  like  that  found  in  a KJF  nota- 
tion for  BASIC,  while  the  horizontal  links  provide 
pedagogical  infomatlcn  sped  tying  relative  diffi- 
culty, analogy,  and  dependency.  The  opinions 
expressed  by  the  horizontal  lit'''-.-,  are  necessarily 
general  and  do  not  always  hold  for  all  students  in 
all  stages  of  learning.  For  f ...  m.ple,  an  arithmetic 
expression  is  generally  a hunter  construct  than  a 
numeric  variable  because  it  often  includes  a 
variable  itself,  but  observation  indicates  that 
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using  n statement  such  an  PRINT  6-*4  tenia  to  be  on 
easier  task  for  a beginner  th'nn  using  PHI NT  II. 

This  implies  that,  ultimately,  the  pedagogical 
relationships  between  concepts  must  r.i  retires  be  a 
function  of  the  student's  state  of  learning  at  the 
time  the  relationships  arc  to  be  used.  We  have 
chosen  not  to  tackle  this  refinement  In  the 
BASICNET  underlying  the  HTP-II  system. 

3-2  List  notation  for  the  BeSICJJSf 

A simplified  version  of  the  list  notation  ve 
use  to  represent  the  portion  of  the  BASICNET  in 
Figure  2 is: 

( COHT ROL ' STRUCTURE  K ( UNCONDITIONAL  CONDITIONAL) ) 
( UNCONDITIONAL  K (END  GOTO  STOP)  H (CONDITIONAL) ) 
(CONDITIONAL  K (TF’TIEN  FOR’ NEXT)) 

( IF 1 THEN  C ("IF"  tOOI EMI  "THEN"  LINENUM)  H 
(FOR ’NEXT)) 

(FOR1 NEXT  C (FOR  NEXT)) 

(FOR  D (NEXT)) 

(BOOLEAN  C (KEXPR  REL  NEXPR) ) 

( HEXFR  K (NUT  IIVAR  SIMARITH) ) 

(NUT  H (IIVAR)  A (SLIT)) 

(NVAR  II  (SIMARITH)  A (SVAR)  S (SVAR)) 

The  A links  specify  that  numeric  literals  are 
analogous  to  string  literals  (SLIT),  and  that 
numeric  variables  are  analogous  to  string  vari- 
ables (SVAR).  The  S link  says  that  NVAR  and  SVAR 
are  similarly  difficult.  (The  information  about 
SLIT  and  SVAR  is  found  in  another  part  of  the 
BASICNET. ) The  notation  here  is  simply  that  used 
to  express  property  lists  in  LISP. 

3.3  The  BASICNET  and  PIP  skills 

After  the  BAEICNST  was  defined,  each  skill  in 
BHP’3  CIN  was  represented  in  terms  of  a subnet. 
First,  the  structure  of  each  skill  was  described, 
in  list  notation  like  the  following  (where  Skill 
1*2  is  "conditional  branch,  comparing  a numeric 
literal  with  a numeric  variable”) : 

(SK0l*2  (IF ’THEN  (BOOLEAN  . (NEXPR  . NUT) 

(NEXPR  . NVAR))) 

Skill  U2  Is  represented  as  an  instance  of  IF’THEH 
(see  Figure  2),  in  which  the  BOOLEAN  component  is 
further  specified  as  consisting  of  the  relation 
between  a numeric  literal  (the  first  NEXPR  com- 
ponent of  the.  BOOLEAN)  and  a numeric  variable 
(the  second  NEXPR).  The  REL  is  left  uninstan- 
tiated, since  SkiU  1*2  does  not  specify  the  kind 
of  comparison  to  te  made  between  the  two.  ThU3, 
any  REL  is  appropriate. 

Skill  1*3  is  "conditional  branch,  comparing  a 
simple  numeric  expression  with  a numeric  variable." 
Its  structure  is 

(GKG>*3  (SK.0>*2  (NEXPR  . SIMARITH))) 

The  notation  is  read  "Skill  1*3  ic  identical  to 
Skill  1*2  except  that  the  first  instance  ox'  iii'.XTR 
should  be  SIMAKITII,"  which  is  exactly  what  the 
English  description  of  the  skill  says.  Skill  1*6 
("conditional  branch,  comparing  two  numeric 
variables" ) is  represented  as 

(SK0>i6  (SKCl*2  ( liHXl’K  . IIVAR))) 


egain  reflecting  the  minimal  difference  between 
the  related  skills. 

3.»*  Skill  set:-. 

Eased  on  ll*c  notation  for  skill  structures, 
skills  were  grouped  together  into  ten  major  skill 
nets  representing  printing,  numeric  assignment, 
string  assignment,  IF-THEKG,  FOR-HEXTS,  etc.  Each 
skill  set  was  fomej  by  sloi-ting  with  a hc.ad  skill, 
not  described  in  terms  of  any  other  skill — like 
Skill  1*2  above — and  all  other  skills  (1*3,  1*6, 
etc.)  described  in  terms  of  it,  or  described  in 
terms  of  other  members  of  the  set.  As  might  be 
expected,  there  was  a degree  of  congruence  between 
the  ten  skill  sets  and  the  technique  groupings  of 
BIP-i. 

3.5  The  SKIIIRHET 

Within  each  skill  set,  pairs  o?  skills  were 
examined  to  find  their  minimal  difference.  If  the 
nodes  by  which  they  differ  are  linked  in  the 
BASICNET,  that  link  was  used  to  define  a relation 
between  the  skills.  If  the  nodes  by  which  two 
skills  differ  do  not  have  a direct  link,  relations 
were  sought  at  increasingly  higher  levels  of  the 
BASICNET. 

For  instance,  since  the  BASICNET  shows  NVAR 
to  be  harder  than  tiLIT,  and  SIMARITH  harder  than 
OTAR,  it  follows  that  Skill  46  is  harder  than  42, 
and  1*3  is  harder  than  46.  The  relationships  de- 
termined in  this  manner  between  all  pairs  of 
skills  comprise  the  SKILLS NET,  a knowledge  repre- 
sentation that  can  be  directly  expressed  in  a CIN 
and  used  for  task  selection.  The  SKILLSNET,  like 
the  BASICNET,  can  be  expressed  in  LISP  property 
list  notation.  (The  underlining  in  the  following 
example  emphasises  the  relationships  being  dis- 
cussed here.) 

(SK0l*2  H (SKOl*!*  SKQl*6  SK0l*7)  A (SKOVf) 

P (SKC03  SKO36  FK039)) 

( SKOl* 3 H (SKd*7  SKC75  CK061) 

P (SK036  SKOUO  LK003  SK005)) 

(SK046  H ( IK 04 ~j  SKC48  SK045)  A (SK048) 

P (SK003  SK036  EK039)) 

The  P links  shewn  here  are  Prerequisite  links; 
like  the  Hardness  links,  these  are  a matter  of 
pedagogical  opinion.  The  P links,  however,  appear 
only  in  the  SKtLISUET,  not  in  the  BASICNET,  and 
express  judgments  that  are  more  specific  to  the 
EIP  ccurse  than  those  expressed  in  the  BASICNET. 

A few  of  the  skills  (e.g.,  those  involving  the  use 
of  built-in  BASIC  functions  such  as  HIT  and  SQR) 
did  not  fall  into  skill  sets  since  they  seemed  not 
to  be  deocribablc  in  terms  of  any  o the r skill. 

These  are  related  within  the  SXILL8NET  only  by 
means  of  P links. 

3.6  BIP-IT  tnsk-r.e lection  procedure 

The  new  task-uelcetlon  procedure  for  the 
1*11*— X L ryot 'si,  designed  to  use  the  relationships 
letve  n skills  expressed  in  tiv  f"r. II.I II'.'ET’,  is 
identical  to  the  technique-*  a;  ed  method  in  its 
o/ernl*  design:  A ret  of  skills  appr  print**  to  the 
student 'a  current  level  el'  understan  ling  is  gencr- 
ft*  I,  " net  of  tanks  using  s -e  of  tlto.e  .-.kills  la 
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wise,  using  the  IV::'.  of  GOOWJGT  GOTO  3c. 
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identi fled,  the  tost  of  those  tasks  (by  so  o cri- 
teria) Jo  |m  r.cntrd,  and  the  student  model  is 
undated  La:  e l on  the  student's  pe l'fu n e:  cc  and 
self-o vahu?. tion  0,,  the  tank,  fh.e  major  difference 
te tween  the  two  methods  Is  that  hy  using  an  ex- 
panded set  of  relations  between  skills  in  the 
SKILLSHET,  the  new  procedure  <;:al  make  more  intel- 
ligent inferences  both  In  updttii’g  the  student 
model  and  in  generating  the  set  of  skills  to  be 
Involved  in  the  student’s  next  task. 

The  following  Is  a simplified  description  of 
the  process  by  which  a task  Is  selected  at  any 
point  during  instruction,  given  the  student's 
state  of  knowledge  of  all  the  skills.  The  pro- 
cedure integrates  a number  of  a priori  reasonable 
pedagogical  heuristics  about  how  to  vary  the 
relative  difficulty  of  tasks  to  optimize  learning 
as  performance  varies  and  about  hoi/  to  teach  a net- 
work of  knowledge  (e.g.,  breadth- first  vs.  depth- 
first  exposure). 

STEP  1:  Create  a set  called  NEED,  consisting 
of  skills  that  will  be  sought  in  the  next  ‘ 
task,  hook  for  "trouble"  skills  first  (those 
in  tasks  that  the  student  quit),  then  for 
analogies  to  learned  skills,  then  for  invsrse- 
pre requisites  of  learned  skills.  As  coon  as 
a group  of  such  skills  is  found,  stop  looking. 

STEP  2;  Remove  from  the  NEED  set  those 
6kills  that  have  unlearned  prerequisites. 

Add  those  skills  to  the  NGTKEADY  set  (which 
may  be  used  later). 


As  an  example  of  tile  Inferences  made  in  the 
generation  of  the  ].7:E!)  ret,  let  us  assume  that 
Skills  1:2  and  Cl  ( "1  OR  . NEXT  loops  with  literal 
as  final  value  of  index")  are  under  consideration. 
Skill  Cl  Is  represented  as 

(fko6i  h (r.y.c62)  p (sko4s)) 

The  Prerequisite  relationship  specified  that  1:2 
must  be  learned  before  a tusk  involving  6 1 can  be 
presented.  The  structure  enforced  by  the  P links 
relating  pairs  of  skills  gives  the  presentation  of 
tasks  some  degree  of  order,  and  is  designed  to 
prevent  too-mpld  progress  or-  drastic  Jumps  in 
difficulty.  The  Hardness  links,  in  contrast,  are 
used  to  facilitate' progress  for-  a student  doing 
veil,  by  allowing  seme  skills  to  be  considered 
"too  easy"  for  inclusion  in  the  NEED  set.  Such 
skills  are  not  inferred  to  be  learned;  they  are 
simply  not  sought  actively  by  the  selection 
algorithm. 

As  an  example  using  the  skills  described  here 
suppose  that  a student  has  successfully  completed 
a task  using  Skill  43,  although  he  has  not  yet 
seen  Skill  42.  (The  fact  that  43  is  harder  than 
42  dees  not  force  42  to  be  presented  first;  only 
P links  force  such  order.)  When  the  task- 
selection  procedure  assembles  the  next  sat  of  NEED 
skills,  it  will  "infer"  that  42  is  now  too  easy  to 
become  part  of  that  set,  since  something  harder 
than  42  has  already  teen  learned. 


STEP  3:  Given  u NEED  set,  find  the  most 
appropriate  task  that  involves  some  of  the 
NEEDed  skills. 

(a)  Assemble  GOODLIST,  those  tasks  that 
have  the*  desired  number  of  NEEDed 
skills,  (j'his  number  increases  if  the 
student  is  consistently  successful, 
decreases  if  he  has  trouble.) 

(b)  If  no  GOODLIST  can  be  created,  make  a 
new  NEED  set  consisting  of  the  pre- 
requisites of  the  skills  in  KOTHEADY. 

If  no  new  NEED  set  can  be  created, 
tlien  the  curriculum  has  been  exhausted; 
otherwise,  GOTO  3a- 

(c)  Find  the  "best"  task:  if  the  student 
is  doing  well,  find  the  task  in 
GOODLIST  that  has  the  fewest  learned 
skills ; if  he  is  progressing  more 
slowly,  find  the  task  with  the  fewest 
unseen  skills.  Remove  the  selected 
task  from  GOODLIST. 

STEP  4:  See  if  the  selected  tusk  is  other- 
wise appropriate. 

(a)  If  none  of  the  skills  in  tlx:  selected 
task  have  unlearned  prerequisites, 
stop  looking  and  present  the  task. 

(END) 

(b)  If  my  skills  have  unsatisfied  pre- 
xcqulAtcu,  reject  the  task  and  add 
those  skills  to  the  NOT  READY  set. 

(c)  If  GGO'.iMfiT  is  exhausted,  change 
(usually  induce)  the  criterial  number 
of  NEE:)  skills,  mid  GOTO  3*1  • Other- 
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Furthermore,  since  42  is  new  considered  too 
easy  to  look  for.  Skill  6.1  can  now  be  sought.  If 
the  student  successfully  completes  a task  Involv- 
ing 6l,  the  student  model  -..-ill  be  updated  to  show 
that  6l  has  been  learned,  and  by  inference,  that 
its  unseen  prerequisite  42  has  also  teen  learned. 
These  kinds  of  inferences  (by  which  skills  can 
reach  too-easy  or  learned  states  without  the 
student  actually  having  seen  them  in  a task)  can 
of  course  be  contradicted  by  direct  observation  or 
by  other  inferences  if  the  student  has  difficulty. 
For  example,  the  unseen  prerequisite  of  a given 
skill  may  change  Its  state  from  "too  easy"  to  "in 
trouble"  If  the  student  quits  (gives  up  on)  a 
task  Involving  the  given  skill.  The  next  task 
selection  would  attempt  to  find  a task  using  that 
prerequisite  skill  in  such  a case. 

3-7  3IP-II  performance 


The  BTP-II  task-selection  procedure  has  been 
implemented  with  parameters  (e.g.,  numbers  of 
skills  sought  Rnd  thresholds  determining  when  an 
unpresented  skill  is  "too  easy"  to  be  Included  in 
the  NEED  set)  that  can  be  changed  readily.  The 
system  can  therefore  te  used  to  explore  the 
effectiveness  of  somewhat  different  pedagogical 
heuristics  for  task  sequencing.  V.'e  used  tnls 
capability  in  conjunction  with  a simulation  system 
ve  developed  to  create  a version  of  RIP- 1 1 that  we 
expected  would  te  effective  for  a range  of  student 
abllt Lies.  Recently,  wo  collected  data  from  a 
group  of  about  27  students  who  tired  this  B1P-IT 
system.  At  this  tlsv*,  the  rlita  have  not  teen 
extensively  analyzed,  but  we  can  provide  a general 
summaiy  mid  report  our  subjective  observations. 


The  students  were  United  to  fifteen  hours  of 
terminal  time  with  DIP.  They  were  presented  with 
(but  did  not  necessarily  complete  successfully)  an 
overuse  of  *t0  tasks,  the  mlulraun  being  21  tasks 
and  the  maximum  63  tasks.  Only  one  student  dropped 
out  of  the  course  without  completing  fifteen  hours 
or  finishing  the  currtculura. 

One  overall  measure  of  the  success  of  seman- 
tic network  CIIl  and  related  task-selection  pro- 
cedure Is  the  relationship  between  number  of 
skills  learned  (according  to  TUP)  and  scores  on  a 
paper-and-pencll  posttest.  The  correlation 
between  these  measures  was  .86,  and  accounts  for 
7U  percent  of  the  variability  in  the  posttest 
scores.  The  students  also  took  a standardized 
test  of  programming  aptitude  prior  to  instruction. 
The  pretest  scores  correlated  significantly  with 
both  the  number  of  skills  learned  (r  = .65)  and 
the  pcsttest  scores  (r  = .59);  however,  in  a 
multiple  regression  of  the  posttest  scores  on  the 
number  of  skills  learned  and  the  pretest  scores, 
only  the  number  of  skills  learned  contributed 
significantly  to  posttest  performance . Number  of 
tasks  presented  to  students  was  independent  of 
both  test  scores  and  number  cf  skills  learned. 

Thus,  the  student  model  maintained  by  the  BIP-II 
system  accurately  reflects  the  acquisition  of  the 
programming  knowledge  required  by  the  pcsttest, 
and  predicts  posttest  performance  independently  of 
the  aptitude  measured  by  the  pretest. 

Our  informal  observations  Indicate  that 
HIP-II  task  sequences  are  substantially  different 
from  those  of  BIP-I.  Host  noticeably,  when  a 
student  dees  well  Initially,  BIP-II  selects  com- 
plex tasks  at  a much  earlier  point  than  they  were 
selected  by  BIP-I.  Students'  reactions  were  ' 
favorable,  even  when  they  spent  considerable  time 
on  these  difficult  tasks  and  then  gave  up.  Task 
selection  following  these  failures  seems  respon- 
sive: BIP-II  selected  ^simpler  tasks  Involving 
some  "not-learned"  skills  that  were  involved  in 
the  task  that  the  student  had  quit.  In  many 
cases,  these  "remedial"  tasks  appear  to  have  been 
too  simple,  given  the  student’s  prior  progress, 
but  most  often  HIP-II  was  in  fact  looking  for  a 
more  challenging  task  and  could  not  find  one  in 
the  curriculum. 

Basides  the  sequencing  "failures"  due  to  the 
limits  of  the  curriculum,  a large  number  of  inade- 
quate task-selection  decisions  were  caused  by  poor 
data  from  the  system’s  solution  checker.  In  these 
cases,  the  student  had  a substantially  correct 
program  that  was  rejected  by  the  checker.  BIP-II 
interprets  the  rejections  as  a sign  that  the 
student  is  ha-ring  difficulty  with  some  of  the 
skills  in  the  task  and  thereby  introduced  errors 
into  the  student  model.  Seme  timer,  students  in 
our  study  became  so  frustrated  by  the  rejection 
of  their  programs  that  they  quit  the  task, 
creating  even  more  severe  errors  in  the  model. 

Disregarding  t.he  difficulties  caused  by  these 
weaknesses  elsewhere  in  the  instructional  system, 
our  initial  evaluation  of  BIP-II ’a  task  selection 
capabilities  ir,  favorable.  However,  It,  is  clear 
that  evaluation  of  the  effects  of  m.iMipuJ.iMng  the 
parameters  of  the  selection  process  will  be  im- 
peded until  the  curriculum  is  expanded  end  the 
solution  checker  is  improved. 


U.  Suej-ih ry  and  conclusions 

We  have  indicated  how  complex  knowledge 
representations  adapted  from  AI  research  can  be 
used  to  describe  a CAI  problem  curriculum  end 
thereby  enable  relatively  sophisticated  individ- 
ualized problem  sequencing.  In  particular,  a CIIJ 
based  on  a semantic-network  representation  pro- 
vides a medium  for  drawing  Indirect  Inferences 
about  what  a student  knov3,  what  he  is  ready  to 
learn,  and  what  task  in  the  curriculum  will  best 
help  him  learn  It.  A network  representation  then 
is  useful  not  only  for  expressing  unambiguous 
relationships  (e.g.,  property-inheritance,  cora- 
ponency),  its  typical  application  in  AI  systems, 
but,  in  addition,  is  one  way  for  systematically 
expressing  opinions  about  the  pedagogical  rela- 
tionships among  the  concepts  and  6kills  a CAI 
system  is  intended  to  tench. 

We  do  not  believe  that  the  BIP-II  system, 
based  on  a CIN  incorporating  a complex  network  of 
skill  relationships,  can  match  a human  tutor's 
ability  to  select  programming  problems  adaptively: 
The  limitations  imposed  by  the  system's  rudimen- 
tary program  checker  Insure  some  extreme  failures, 
but,  beyond  this,  the  SKILLSKET  and  the  inferences 
that  use  it  only  approximate  the  flexibility  of 
which  a tutor  is  capable.  Nonetheless,  the  more 
evolved  CM  makes  it  possible  for  tutorial  CAI  in 
technical  subjects  to  individualize  student  exper- 
ience effectively  across  a range  of  student 
abilities  and  instructional  goals. 
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